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The 

Evaluation 

System 


The  evaluation  system  is  referred  to  as  the 
Three  Phase  System  for  Statewide  Evaluation  of 
Occupational  Education  Programs.  The  goals  of 
this  system  are: 

To  promote  and  assist  with  the  development  of 
quality  occupational  programs  in  local  education 
agencies. 

To  foster  maximum  utilization  and  accountability  of 
State  and  Federal  funds  allocated  to  occupa¬ 
tional  programs. 

To  provide  the  Department  of  Adult,  Vocational 
and  Technical  Education  with  necessary  data  for 
statewide  planning  of  occupational  programs. 


The  evaluation  system  has  three  rather  distinct 
but  interrelated  phases.  As  a  team  member,  your 
contribution  will  be  to  the  third  phase. 

Phase  I  involves  evaluation  and  planning  of  the 
local  education  agency’s  occupational  program. 
The  staff  of  the  local  agency  is  responsible  for 
this  Phase  which  culminates  in  the  preparation 
and  submission  of  a  One  and  Five  Year  Plan  for 
occupational  education  to  the  Department  of 
Adult,  Vocational  and  Technical  Education. 

Phase  II  involves  the  annual  assessment  of  each 
agency’s  local  plan  by  staff  of  the  DAVTE’s  Pro¬ 
gram  Approval  and  Evaluation  Section.  Each  plan 
is  evaluated  in  light  of  local  resources,  student 
needs,  human  resource  needs,  and  agency  po¬ 
tential.  Such  things  as  population  characteristics 
and  past  performance  are  taken  into  account 
during  this  Phase. 

Phase  III  involves  the  on-site  visitation  each  year 
of  a  portion  of  the  agencies  with  approved  plans. 
On-site  evaluations  utilize  the  team  approach  to 
review  information  provided  by  the  local  education 
agency;  to  interview  staff,  students  and  com¬ 
munity  representatives;  and  to  prepare  a  report 
of  conclusions  and  recommendations  for  improve¬ 
ment. 


Focus 
of  the 
Evaluation 


Several  aspects  of  an  educational  institution 
constitute  the  total  occupational  education  pro¬ 
gram.  They  include: 

Occupational  Information  in  Kindergarten  to 
Grade  8. 

Occupational  Orientation  in  Grades  9  and  1 0. 

Occupational  Experiences  in  regular  secondary 
schools,  area  vocational  centers,  community 
colleges,  and  state  agencies  such  as  Corrections, 
Mental  Health?  and  Children  and  Family  Services. 

Depending  upon  the  type  of  local  education 
agency  being  visited,  the  evaluation  team  inter¬ 
views  a  variety  of  individuals  within  the  agency 
and  representatives  from  external  community 
agencies  who  are  important  to  student  services, 
education,  job  placement,  and  employment  of 
occupational  students. 


Program 

Components 


The  on-site  evaluation  focuses  on  the  total 
program  of  occupational  education  provided  to 
students.  The  team  centers  its  attention  on  six 
program  components.  These  are  used  to  analyze 
the  total  learning  opportunities  and  support  serv¬ 
ices  for  occupational  students.  The  six  program 
components  also  form  the  body  of  the  evalua¬ 
tion  report.  They  are: 

Planning  and  Evaluation  —  Evidence  is 
gathered  by  the  team  to  determine  the  adequacy 
of  the  local  education  agency’s  system  for  estab¬ 
lishing  total  program  goals,  identifying  needed 
occupational  programs,  and  designing  instruc¬ 
tion.  Additionally,  the  local  education  agency’s 
system  of  evaluating  both  the  processes  of  in¬ 
struction  and  the  outcomes  or  impact  of  programs 
is  assessed. 

Occupational  Programs  —  The  team  analyzes 
the  quality  of  each  occupational  program  through 
interviews  and  document  review  to  assess  the 
availability  of  programs  to  all  students  as  well  as 
program  scope  and  course  sequence.  Addi¬ 
tionally,  the  team  will  assess  the  adequacy  of 
internal  resource  utilization  within  various  pro¬ 
grams. 

Student  Services  —  The  various  support  or 
ancillary  services  provided  to  occupational  stu¬ 
dents  are  reviewed  by  the  evaluation  team. 
Services  such  as  guidance  and  counseling, 
placement,  and  student  testing  are  focused  on 
through  the  interview  and  document  review  pro¬ 
cess.  Services  for  special  populations  such  as  the 
disadvantaged,  handicapped,  minority  groups  and 
women  are  also  included  in  this  component. 


Personnel  —  The  team  reviews  personnel 
qualifications  in  light  of  personnel  development 
needs,  and  analyzes  the  type  and  extent  of  par¬ 
ticipation  in  various  personnel  development  ac¬ 
tivities  such  as  professional  organizations, 
workshops,  and  courses. 

Program  Management  —  The  team  looks  at  the 
way  occupational  programs  are  managed.  This 
includes  both  the  administrative  structure  as  well 
as  the  operational  effectiveness  of  the  structure. 
This  component  includes  management  at  all 
levels  of  the  local  education  agency  that  con¬ 
tributes  to  the  delivery  of  instruction  and  services 
to  occupational  students. 

Community  Resources  —  The  team 
focuses  on  this  component  to  determine  to  what 
extent  the  local  education  agency  has  involved 
community  representatives  and  utilized  com¬ 
munity  resources  in  the  development,  operation 
and  evaluation  of  occupational  programs.  The  use 
of  advisory  committees,  cooperation  with  other 
educational  and  service  agencies  and  coopera¬ 
tive  education  pursuits  are  examples  of  areas  to 
be  reviewed. 


The 

Team 


The  Department  of  Adult,  Vocational  and  Tech¬ 
nical  Education  is  careful  to  assure  that  on-site 
evaluations  are  conducted  by  competent,  well- 
balanced  teams  from  outside  the  immediate  locale 
of  the  local  education  agency.  In  addition  to  a 
team  leader,  team  members  representative  of  three 
areas  are  selected  for  each  evaluation.  They  are: 

1.  Business,  industry  or  labor  representatives 
who  have  a  concern  for  and  an  interest  in  occu¬ 
pational  education. 

2.  Practicing  educators  who  have  demonstrated 
teaching,  administrative  or  support  service  exper¬ 
tise  in  occupational  education. 

3.  Advanced  students  and  graduates  of  occu¬ 
pational  programs. 

The  number  of  team  members  in  each  of  these 
three  categories  depends  upon  size  and  other 
features  of  the  local  education  agency  being 
evaluated. 


Review  of 

Pre-Visitation 

Information 


Team  members  are  provided  with  materials  for 
study  before  the  on-site  visitation.  The  local 
agency  and  the  state  office  send  each  team 
member  copies  of: 

The  Local  Plan.  This  is  the  document  which  was 
prepared  by  the  local  agency  staff  (in  Phase  I) 
and  approved  by  state  office  staff  (in  Phase  II). 

The  Local  Plan  tells  what  the  agency  is  doing  for 
occupational  students. 

School  and  Community  Data.  This  form  pro¬ 
vides  basic  information  about  the  community  in 
general,  the  local  education  agency  and  its 
respective  occupational  programs. 

Preliminary  Evaluation  Information  Faculty 
Form.  Several  weeks  before  the  on-site  visita¬ 
tion,  teachers,  counselors,  and  administrators 
complete  a  questionnaire  about  the  six  program 
components.  These  responses  are  tabulated  and 
sent  to  team  members  for  pre-study  and  for  use 
during  the  visitation. 

Preliminary  Evaluation  Information  Student 
Form.  A  sample  of  occupational  students  com¬ 
pletes  a  questionnaire  about  occupational  pro¬ 
grams  and  related  services.  Results  are  tabulated 
and  also  sent  to  team  members. 


Orientation 
of  Team 


The  visitation  begins  with  an  orientation 
meeting.  This  meeting  is  conducted  by  an 
experienced  team  leader  who  explains  the  pro¬ 
cedures  to  be  followed.  The  orientation  meeting 
involves: 

Becoming  acquainted  with  other  team  members. 

Reviewing  the  philosophy  of  the  evaluation 
system  and  the  major  components  of  the 
evaluation. 

Reviewing  available  materials. 

Preparing  for  interviews. 

Preparing  for  report  writing  sessions. 

Understanding  the  schedule  of  events  for  the 
entire  visitation. 

The  team  leader  providing  interview  schedules  for 
each  team  member. 


Interviews 


V 


y 


Much  of  your  time  as  a  team  member  will  be 
spent  interviewing  various  individuals.  The  kinds 
of  individuals  who  will  be  interviewed  are: 

Administrators  (both  occupational  and  general) 

Instructional  Personnel  (all  occupational  staff  plus 
a  sample  of  others) 

Guidance  Personnel 

Governing  Board  Members 

Business,  Government  and  Industry  Personnel 
(advisory  committee,  employers  and  others) 

Students  (occupational  and  others) 

A  Team  Member  Handbook  will  be  mailed  to 
you  along  with  pre-visitation  information.  This 
handbook  provides  an  orientation  to  interviewers 
and  also  provides  a  detailed  structure  to  aid  you 
in  the  interviewing  process. 


Writing 

Sessions 


The  team  spends  late  afternoons  and  evenings 
in  group  writing  sessions.  The  team  leader  uses 
a  variety  of  approaches  for  drafting  the  evalua¬ 
tion  report.  The  report  will  have  a  section  for  each 
of  the  six  program  components  and  will  be  a 
composite  of  the  views  of  all  the  team  members. 

During  the  writing  sessions,  team  members  use 
the  information  sources  provided  before  the 
evaluation,  materials  about  programs  and  courses 
provided  by  interviewees,  interview  notes,  other 
materials  and  their  own  analysis  to  suggest 
statements  for  the  report.  The  team  leader  uses 
these  to  phrase  conclusions  and  recommenda¬ 
tions. 

Most  of  the  time,  the  team  leader  does  the 
actual  writing.  The  team  members’  job  will  be  to 
offer  findings  and  ideas  for  improvement.  The 
team  leader  may  ask  team  members  to  work  in 
small  groups  to  prepare  preliminary  drafts  of 
some  parts  of  the  report. 


Summary 

Conference 


The  on-site  evaluation  ends  with  a  Summary 
Conference.  The  conference  is  attended  by  the 
leadership  of  the  local  education  agency  and 
community  representatives.  It  is  conducted  by 
the  Department  of  Adult,  Vocational  and 
Technical  Education  Regional  Director  who  will 
join  the  team  on  its  final  afternoon  in  the  com¬ 
munity.  The  team  leader  represents  the  team  and 
presents  the  evaluation  report  and  receives  sug¬ 
gestions  for  possible  last  minute  revision. 


Summary 


Evaluations  are  very  important  to  the  improve¬ 
ment  of  ongoing  occupational  programs.  The 
DAVTE  hopes  that  you  will  accept  the  invitation  to 
serve  on  a  team  and  help  improve  occupational 
education  for  students. 

If  you  agree  to  serve,  you  will  receive  ma¬ 
terials  mentioned  earlier  in  this  publication  and 
materials  regarding  mechanics,  such  as  motel  or 
hotel  accommodations.  You  will  be  expected  to 
do  some  pre-visit  study,  to  attend  the  orientation 
meeting  and  to  engage  in  interview  and  report 
preparation  sessions  with  the  team.  Approxi¬ 
mately  four  weeks  after  the  on-site  visitation,  you 
will  receive  a  copy  of  the  evaluation  report  you 
helped  prepare. 
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